Systems and methods for interacting with a computing device using gaze information

ABSTRACT

Techniques for interacting with a first computing device based on gaze information are described. In an example, the first computing device captures a gaze direction of a first user of the first computing device by using an eye tracking device. The first computing device displays a representation of a second user on a display of the first computing device. Further, the first computing device receives from the first user, communication data generated by an input device. The first computing device determines if the gaze direction of the first user is directed to the representation of the second user. If the gaze direction of the first user is directed to the representation of the second user, the first computing device transmits the communication data to a second computing device of the second user.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority to Provisional Application No.62/561,585, entitled “SYSTEMS AND METHODS FOR INTERACTION WITH ACOMPUTING DEVICE USING GAZE INFORMATION,” filed Sep. 21, 2017, which isassigned to the assignee hereof and expressly incorporated by referenceherein in its entirety.

TECHNICAL FIELD

The present disclosure generally relates to systems and methods forinteracting with a computing device and in particular, to systems andmethods for interacting with a computing devices using gaze information.

BACKGROUND

Interaction with computing devices is a fundamental action in today'sworld. Computing devices from personal computers, to mobile devices, towearable headsets, to smart watches are found throughout daily life. Thesystems and methods for interacting with such devices defines how theyare used, and what they are used for.

Traditionally computing devices have been interacted with using“contact” based interaction techniques. For example the computer mouse,keyboard, and touch screen can all be considered contact based inputtechniques as they require a user to physically contact something tointeract with the computing device.

Recently, advances in eye tracking technology have made it possible tointeract with a computing device using a person's gaze information. Inother words, the location on a display the user is gazing at. Thisinformation can be used for interaction solely, or in combination with acontact based interaction technique.

Previously proposed interaction techniques using gaze information can befound in U.S. Pat. No. 6,204,828, United States Patent Application20130169560, U.S. Pat. No. 7,113,170, United States Patent Application20140247232, and U.S. Pat. No. 9,619,020. The full specification ofthese patents and applications are herein incorporated by reference.

Gaze based interaction techniques offer advantages over traditionalcontact only interaction techniques, as a person's gaze is faster thanmoving a physical limb, and intent can easily be communicated using gazeby merely looking at something.

The present disclosure seeks to provide new and improved systems andmethods for interacting with a computing device using gaze information.

SUMMARY

Thus, an object of the present disclosure is to provide improved systemsand methods for interacting with a computing device using gazeinformation. This and other objects of the present disclosure will bemade apparent from the specification and claims together with appendeddrawings.

According the present disclosure, there is provided a system and methodfor interacting with a computing device using gaze information. Whereinthe system contains a display, an eye tracker, and a processor foraltering or otherwise controlling the contents of the display.

An eye tracker is a device which is able to approximate a user's gazelocation on a display. Any form of eye tracker will suffice for thepresent disclosure, however the common form of an eye tracker is aninfrared, image based system. In such a system, infrared illumination isdirected toward a user's eye or eyes, images are then captured of theeye or eyes and based on the user's pupil and reflection of the infraredillumination, the eye tracker (or a computing device connected to theeye tracker) may approximate a user's gaze direction. The form of an eyetracker is not the subject of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

A further understanding of the nature and advantages of variousembodiments may be realized by reference to the following figures.

FIG. 1 illustrates an eye tracking system, according to an embodiment ofthe present disclosure.

FIG. 2 illustrates an example of an image of an eye capture by an imagesensor, according to an embodiment of the present disclosure.

FIG. 3 illustrates a block diagram of a specialized computer system,according an embodiment of the present disclosure.

FIG. 4 illustrates a computer system for managing communication based ongaze information, according an embodiment of the present disclosure.

FIG. 5 illustrates a flow implemented by a first computing device tocommunicate with a second computing device based on gaze information,according an embodiment of the present disclosure.

FIG. 6 illustrates a flow for managing communication based on gazeinformation, according an embodiment of the present disclosure.

FIG. 7 illustrates a computer system for managing the presentation ofcontent based on gaze information, according an embodiment of thepresent disclosure.

FIG. 8 illustrates a flow implemented by a first computing device tomanage the presentation of content based on gaze information, accordingan embodiment of the present disclosure.

FIG. 9 illustrates a flow for managing to manage the presentation ofcontent based on gaze information, according an embodiment of thepresent disclosure.

DETAILED DESCRIPTION

Remote Sharing

According to a first aspect of the present disclosure, a method forsharing information is provided wherein two or more users are usingseparate computing devices. Each computing device is equipped with aneye tracker according to the present disclosure.

A location is provided on each user's display where when the users areboth gazing at the same spot, information can be shared. This isachieved using a method approximating the following, where User 1 isinteracting with Computer 1, and User 2 is interacting with Computer 2:

-   -   1. Computer 1 displays a designated sharing location on its        display.    -   2. Computer 2 displays a designated sharing location on its        display.    -   3. User 1 gazes at the designated sharing location on Computer        1.    -   4. User 1 attempts to share information by moving the        information to the designated sharing location (or another        predetermined location). This sharing of information may take        place using a traditional “drag and drop” movement whereby        information such as a file, or document, is moved using a        contact input method across a display and “dropped” by releasing        the contact input method at the desired location.    -   5. If User 2 is gazing at the designated sharing location on        Computer 2, the information is transmitted from Computer 1 to        Computer 2.    -   6. If User 2 is not gazing at the designated sharing location on        Computer 2, the information is not transmitted from Computer 1        to Computer 2.

The designated sharing location may be the same on both computingdevices, or it may be different. It may be indicated by a graphicalmarker, or it may be an invisible location. Additionally, any number ofcomputing devices and user's may interact using this aspect of thepresent disclosure.

In a further improvement, it is sufficient only that User 2 gazes at thedesignated sharing location in order to accept transmission of theinformation. It is further sufficient that User 1 gaze at the designatedsharing location, and is able to look away during transmission of theinformation.

Presence Window

According to a second aspect of the present disclosure, there isprovided a location on a display that alters based on whether a user, orusers, is present in front of a computing device.

The presence of the user is preferably determined by an eye tracker.

When a first user is determined to be present in front of a firstcomputing device, and concurrently a second user is determined to bepresent in front of a second computing device, information displayed onthe display may be altered. For example a specific window or area mayappear on the display, or the properties of an existing item (such asbrightness or color) on the display may change. Further, oralternatively, an audible or other feedback may sound. This window maybe used for text or video based chatting, sending and receiving files,sharing information or anything else possible on the computing device.

Head Position Camera

According to a third aspect of the present disclosure, there is provideda system and method for a first user to control the camera of acomputing device being used by a second user.

According to this aspect, the first user moves his or her head in orderto control the camera of a computing device used by a second user.

By way of example, consider a scenario where two users are conducting avideo call. The first user on the first computing device moves his orher head, and the image transmitted by the second computing device ofthe second user is altered. For example, the first user may tilt theirhead to the right, causing the perspective of the image transmitted fromthe second computing device to move to the left, and vice-versa.

In a further improvement, the altering of the image may be based on gazedirection rather than head orientation.

The altering of the image transmitted may be achieved by either physicalmanipulation of the camera, or by processing of the image captured bythe camera, as would be readily understood by a person of skill in theart.

Eye Activated Dialogue

According to a fourth aspect of the present disclosure, as furtherdescribed in connection with FIGS. 4-6, there is provided a system andmethod for a first user of a first computing device to communicate witha second user of a second computing device using gaze as a trigger tothe communication.

According to this aspect of the present disclosure, representations ofusers are displayed on a display. Each representation representing auser using a computing device. Preferably these users are remote fromone another.

A first user, using a first computing device having representations ofat least one other user on the display, may gaze at the representationof a user. An eye tracker, or the computing device, may determine thatthe first user is gazing towards the representation of the second user.The first user may then direct an interaction toward the second user bygazing at the second user's representation.

In a further improvement, the second user may be alerted to the fact thefirst user is gazing at their representation by some form of visual, oraudible, notification

By way of example, the first user may gaze at the representation of thesecond user and speak words. These spoken words are transmitted to thesecond user's computing device and emitted by a speaker connected to thesecond user's computing device. Alternatively to spoken words, textcommunication, file transmittal, or any other form of interaction couldbe used.

This communication may be initiated simply by the first user gazing atthe second user's representation, or alternatively it may only beinitiated if both users are gazing at their counterparts respectiverepresentations on their computing devices.

Further, the first user may have access to a pointing device. In oneexample, the pointing device is used to move a file or a document on thedisplay to the representation of the second user by means of thepointing device. In another example, the words are transmitted to thesecond user's computing device only if, additionally, an action istriggered by the pointing device during the time period when the gazedirection of the first user is directed to the representation of thesecond user. The pointing device may be a mouse, a pointer, a joystickor similar. The action triggered by the pointing device may relate topressing a button, performing a specific movement or similar.

Any form of communication may take place, from verbal, to text, tovideo, to transmittal of files.

Responsive Portraits

According to a fifth aspect of the present disclosure, there is provideda system and method for a representation of a user displayed on adisplay to respond to another user's gaze being directed to therepresentation.

In this aspect of the present disclosure, a representation of a seconduser is displayed on the display of a first user's computing device.When the first user gazes toward this representation, the representationalters in response.

In a further improvement, the a representation of the first user may bedisplayed on the display of the second user's computing device, and whenboth users are gazing towards the representation of the other user ontheir computing device, the representations may further alter.

Any reference to representations or the like in this specification areintended to include any form of graphical representation conceivable,including but not limited to, a photographic representation, an animatedrepresentation, a cartoon-like representation, a line-drawingrepresentation, a shadow-like representation, or any alternativerepresentation (for example an animal or other character).

Only one user requires a computing device equipped with an eye trackerfor this aspect of the present disclosure.

Watch Together

According to a sixth aspect of the present disclosure, as furtherdescribed in connection with FIGS. 7-9, there is provided a system andmethod for multiple users to view content simultaneously on remotedevices.

According to this aspect of the present disclosure, a first user of afirst computing device shares content (such as video, a shared virtualspace, audio, video game graphics, text, multimedia pages or the like)with a second user of a second computing device. Both computing devicesare equipped with an eye tracker, and thus are capable of tracking auser's gaze relative to a display of the computing device.

The two computing devices synchronize that both users are viewing thesame piece of content, based on their gaze activity relative to thedisplay.

In a further improvement, the content may be created collaboratively bythe users, for example a shared drawing. In this way each user maycreate the drawing, as would be understood by a person of skill in theart any manner of traditional computer based drawing methods may beused, and the user's gaze information can be used by the other usersduring the collaborative drawing process.

Highlight Reel

According to a seventh aspect of the present disclosure, there isprovided a system and method for determining a user's reaction tocontent at the time the user viewed the content.

According to this aspect of the present disclosure, when a user isviewing content such as a video, on a computing device equipped with aneye tracker, the computing device registers the user's reaction tospecific portions of the content and records that reaction.

A user's reaction may be based on gaze, or it may be based on facialfeatures.

The computing device may analyze the user's reaction to determine if thereaction is positive, negative, neutral or the like. Various degrees ofreaction would be obvious to a person of skill in the art.

These reactions may be compiled and added to the content, for example inthe case of a video, graphical representations of a user's reaction(such as a smiling face for a positive reaction) may be added below thevideo at the time in the video the user reacted. These representationscould appear anywhere in the video, or separately to the video. Forexample graphical representations of a user's reaction could appear atthe gaze position of the user on the video, or at an averaged gazelocation of the user over a predetermined period of time.

Instead of a graphical representation, a user's reaction may simply bedisplayed as the actual image, images, or video, recorded by the eyetracker.

User reactions may further be compiled into their own video.

Face to Avatar

According to an eight aspect of the present disclosure, there isprovided a system and method for translating a user's facial expressionsto a graphical representation of the user.

According to this aspect of the present disclosure, a graphicalrepresentation of a user such as an avatar is altered based on imagescaptured by an eye tracker. These images may contain items such asfacial expressions, and gaze movements.

The graphical representation may alter to match these facial expressionsand or gaze movements exactly, or in an exaggerated fashion.

The graphical representations may be in any form, including but notlimited to, a photographic representation, an animated representation, acartoon-like representation, a line-drawing representation, ashadow-like representation, or any alternative representation (forexample an animal or other character).

In a further improvement, the avatar may react to a user's gaze. Forexample when a user gazes at the avatar's position on the display, theavatar looks directly back at the user. Further, for example, if theuser gazes to the side of the avatar, the avatar also gazes to the side.

In a further improvement, the avatar may be seen to be performing itsown tasks on the display, or no tasks at all, until it is gazed at. Forexample the avatar may be animated such that it appears to be moving,and then reacts to the user's gaze when the user's gaze is on or nearthe graphical representation.

Collaborative Presence

According to a ninth aspect of the present disclosure there is provideda system and method for multiple users using different computing devicesto view one another's gaze point relative to the same content.

According to this aspect, a first user on a first computing device withan eye tracker views content, and a second user on a second computingdevice with an eye tracker views the same content. The eye tracker ofeach computing device determines the user's gaze direction, and thecomputing device transmits that direction to the other computing device.The other computing device then displays the gaze direction on itsdisplay.

In this way, a first user may see where a second user is gazing inrelation to the same content the first user is gazing at.

This is particularly beneficial in a collaborative environment wheremultiple users are viewing or altering the same content.

Microphone Control

In a tenth aspect of the present disclosure there is provided a systemand method for controlling a microphone or sound recording device of acomputing device based on a user's gaze.

According to this aspect, when a user gazes towards or near apredetermined area, the microphone or sound recording device of thecomputing device is enabled.

The predetermined area may be on a display (for example an icon or othergraphic item), or off a display (for example a keyboard, or logo of thecomputer device maker).

In a further improvement, the microphone or sound recording device mayonly be enabled when it is determined that a user is present in front ofthe computing device, in this improvement the user's gaze direction neednot be used. This determination of presence may be performed based on auser's gaze direction, a user's facial features, a user's head locationor orientation, or any other analysis of an image captured by a cameraconnected to the computing device.

According to this aspect, a microphone or other sound recording devicemay only be enabled and able to capture audio when a user desires it tobe so, or at least only enabled when a user is actually present by thecomputing device. When the microphone or sound recording device isenabled, it may be continuously recording or transmitting audio, or itmay intermittently record or transmit audio.

Eye Tracking

FIG. 1 shows an eye tacking system 100 (which may also be referred to asa gaze tracking system), according to an embodiment. The system 100comprises illuminators 111 and 112 for illuminating the eyes of a user,and a light sensor 113 for capturing images of the eyes of the user. Theilluminators 111 and 112 may for example be light emitting diodesemitting light in the infrared frequency band, or in the near infraredfrequency band. The light sensor 113 may for example be a camera, suchas a complementary metal oxide semiconductor (CMOS) camera or a chargedcoupled device (CCD) camera.

A first illuminator 111 is arranged coaxially with (or close to) thelight sensor 113 so that the light sensor 113 may capture bright pupilimages of the user's eyes. Due to the coaxial arrangement of the firstilluminator 111 and the light sensor 113, light reflected from theretina of an eye returns back out through the pupil towards the lightsensor 113, so that the pupil appears brighter than the iris surroundingit in images where the first illuminator 111 illuminates the eye. Asecond illuminator 112 is arranged non-coaxially with (or further awayfrom) the light sensor 113 for capturing dark pupil images. Due to thenon-coaxial arrangement of the second illuminator 112 and the lightsensor 113, light reflected from the retina of an eye does not reach thelight sensor 113 and the pupil appears darker than the iris surroundingit in images where the second illuminator 112 illuminates the eye. Theilluminators 111 and 112 may for example take turns to illuminate theeye, so that every second image is a bright pupil image, and everysecond image is a dark pupil image.

The eye tracking system 100 also comprises circuitry 120 (for exampleincluding one or more processors) for processing the images captured bythe light sensor 113. The circuitry 120 may for example be connected tothe light sensor 113 and the illuminators 111 and 112 via a wired or awireless connection. In another example, circuitry 120 in the form ofone or more processors may be provided in one or more stacked layersbelow the light sensitive surface of the light sensor 113.

FIG. 2 shows an example of an image of an eye 200, captured by the lightsensor 113. The circuitry 120 may for example employ image processing(such as digital image processing) for extracting features in the image.The circuitry 120 may for example employ pupil center cornea reflection(PCCR) eye tracking to determine where the eye 200 is looking. In PCCReye tracking, the processor 120 estimates the position of the center ofthe pupil 210 and the position of the center of a glint 220 at the eye200. The glint 220 is caused by reflection of light from one of theilluminators 111 and 112. The processor 120 calculates where the user isin space using the glint 220 and where the user's eye 200 is pointingusing the pupil 210. Since there is typically an offset between theoptical center of the eye 200 and the fovea, the processor 120 performscalibration of the fovea offset to be able to determine where the useris looking. The gaze directions obtained from the left eye and from theright eye may then be combined to form a combined estimated gazedirection (or viewing direction). As will be described below, manydifferent factors may affect how the gaze directions for the left andright eyes should be weighted relative to each other when forming thiscombination.

In the embodiment described with reference to FIG. 1, the illuminators111 and 112 are arranged in an eye tracking module 110 placed below adisplay watched by the user. This arrangement serves only as an example.It will be appreciated that more or less any number of illuminators andlight sensors may be employed for eye tracking, and that suchilluminators and light sensors may be distributed in many different waysrelative to displays watched by the user. It will be appreciated thatthe eye tracking scheme described in the present disclosure may forexample be employed for remote eye tracking (for example in a personalcomputer, a smart phone, or integrated in a vehicle) or for wearable eyetracking (such as in virtual reality glasses or augmented realityglasses).

Miscellaneous

The person skilled in the art realizes that the present disclosure is byno means limited to the preferred embodiments described above. On thecontrary, many modifications and variations are possible within thescope of the appended claims. For example, the person skilled in the artrealizes that the eye/gaze tracking methods described herein may beperformed by many other eye/gaze tracking systems than the exampleeye/gaze tracking system 100 shown in FIG. 1, for example using multipleilluminators and multiple cameras.

Additionally, variations to the disclosed embodiments can be understoodand effected by those skilled in the art in practicing the claimeddisclosure, from a study of the drawings, the disclosure, and theappended claims. In the claims, the word “comprising” does not excludeother elements or steps, and the indefinite article “a” or “an” does notexclude a plurality. The division of tasks between functional unitsreferred to in the present disclosure does not necessarily correspond tothe division into physical units; to the contrary, one physicalcomponent may have multiple functionalities, and one task may be carriedout in a distributed fashion, by several physical components incooperation. A computer program may be stored/distributed on a suitablenon-transitory medium, such as an optical storage medium or asolid-state medium supplied together with or as part of other hardware,but may also be distributed in other forms, such as via the Internet orother wired or wireless telecommunication systems. The mere fact thatcertain measures/features are recited in mutually different dependentclaims does not indicate that a combination of these measures/featurescannot be used to advantage. Method steps need not necessarily beperformed in the order in which they appear in the claims or in theembodiments described herein, unless it is explicitly described that acertain order is required. Any reference signs in the claims should notbe construed as limiting the scope.

Specialized Computer System

FIG. 3 is a block diagram illustrating a specialized computer system 300in which embodiments of the present disclosure may be implemented. Thisexample illustrates specialized computer system 300 such as may be used,in whole, in part, or with various modifications, to provide thefunctions of components described herein.

Specialized computer system 300 is shown comprising hardware elementsthat may be electrically coupled via a bus 390. The hardware elementsmay include one or more central processing units 310, one or more inputdevices 320 (e.g., a mouse, a keyboard, eye tracking device, etc.), andone or more output devices 330 (e.g., a display device, a printer,etc.). Specialized computer system 300 may also include one or morestorage device 340. By way of example, storage device(s) 340 may be diskdrives, optical storage devices, solid-state storage device such as arandom access memory (“RAM”) and/or a read-only memory (“ROM”), whichcan be programmable, flash-updateable and/or the like.

Specialized computer system 300 may additionally include acomputer-readable storage media reader 350, a communications system 360(e.g., a modem, a network card (wireless or wired), an infra-redcommunication device, Bluetooth™ device, cellular communication device,etc.), and working memory 380, which may include RAM and ROM devices asdescribed above. In some embodiments, specialized computer system 300may also include a processing acceleration unit 370, which can include adigital signal processor, a special-purpose processor and/or the like.

Computer-readable storage media reader 350 can further be connected to acomputer-readable storage medium, together (and, optionally, incombination with storage device(s) 340) comprehensively representingremote, local, fixed, and/or removable storage devices plus storagemedia for temporarily and/or more permanently containingcomputer-readable information. Communications system 360 may permit datato be exchanged with a network, system, computer and/or other componentdescribed above.

Specialized computer system 300 may also comprise software elements,shown as being currently located within a working memory 380, includingan operating system 384 and/or other code 388. It should be appreciatedthat alternate embodiments of specialized computer system 300 may havenumerous variations from that described above. For example, customizedhardware might also be used and/or particular elements might beimplemented in hardware, software (including portable software, such asapplets), or both. Furthermore, connection to other computing devicessuch as network input/output and data acquisition devices may alsooccur.

Software of specialized computer system 300 may include code 388 forimplementing any or all of the function of the various elements of thearchitecture as described herein. For example, software, stored onand/or executed by a specialized computer system such as specializedcomputer system 300, can provide the functions of components of thedisclosure such as those discussed above. Methods implementable bysoftware on some of these components have been discussed above in moredetail

Throughout this document references have been made to an “eye tracker”or similar. Although the present disclosure has been described withreference to eye or gaze tracking, it is intended that the presentdisclosure function with any form of image based capture system. Forexample the “eye tracker” does not necessarily need to determine auser's gaze direction, but could determine just a user's facialfeatures, expressions or even simply whether a user is present in frontof a computing device.

Any reference to a “computing device” or similar is intended to includeany form of computing, from desktop based personal computing, towearable devices such as Virtual Reality or Augmented Reality headsets,to portable devices such as mobile phones or tablets.

Any reference to a “display” or similar is intended to include any formof display, such as a head-mounted display, a virtual reality display,an augmented reality display or a combination thereof.

Eye Activated Dialogue

FIGS. 4-6 illustrate example embodiments for eye activated dialogues. Inan example, users operates computing device to communicate. Gazeinformation of the users can be used to trigger and manage thecommunications.

FIG. 4 illustrates a computer system for managing communication based ongaze information, according an embodiment of the present disclosure. Asillustrated, the computer system includes a first computing device 420of a first user 410 and a second computing device 470 of a second user460. The first computing device 420 and the second computing device 470are communicatively coupled over a data network (not shown in FIG. 4),such as the Internet or a local area network, to exchange communicationinformation. The exchange of the communication information, includingtriggers for starting and ending the transmission of the communicationinformation and alerts about the communication may depend on the gaze ofthe first user 410 and/or the second user 460.

Optionally, the computer system further includes a communication server450 that manages certain aspects of the communication between thecomputing devices 420 and 470. The use of the communication server 450is further described herein below and operations thereof are furtherillustrated in FIG. 7.

In an example, the first computing device 420 includes a number ofcomputing components, such as a processor, a memory, a display (shown asa display 426 in FIG. 4), input/output (I/O) interfaces, networkinterfaces, and other computing components for receiving input of thefirst user 410 and communicating with the second computing device 470.For instance, the first computing device 420 can be a mobile device,tablet, a laptop, a desktop computer, a wearable device (e.g., a virtualreality device, an augmented reality device; in such a case, the display426 is a head-mounted display) or any other suitable user device. Inaddition, the first computing device 420 includes (e.g., integrates orinterfaces with) an eye tracking device 422 having similar components asthe eye tacking system 100 of FIG. 1. The eye tracking device 422 isused to track a gaze 424 of the first user 410 on the display 426 andaccordingly generate gaze information. The gaze information includes,among other data, a gaze direction of the gaze 424.

In an example, the first computing device 420 executes a communicationapplication that facilitates the exchange of the communicationinformation with the second computing device 470. The communicationapplication may support the exchange of text data, audio data, imagedata, video data, files, documents, executable codes or other digitalcommunication data (depending on the input device, as explained hereinbelow).

As part of supporting the communication, the communication applicationcan display a representation 430 of the second user 460 on the display426 (e.g., in a graphical user interface (GUI) rendered on the display426). The representation can be a static graphical object representingthe second user 460, such as an image or an avatar, or can be a dynamicgraphical object such as a live video of the second user 460, where thelive video is received as a video stream from the second computingdevice 470 over the data network. Generally, the communicationapplication presents the representation 430 at particular location onthe GUI of the display 426 and maintains the location information (e.g.,pixel coordinates) of the representation 430.

The communication application may also interface with the eye trackingdevice 422 (or an eye tracking application executed by the eye trackingdevice 422) based on an application programming interface (API).Accordingly, the communication application can receive certain gazeinformation from the eye tracking device 422 including, for instance,the gaze direction and/or location relative to the display 426 (the gazelocation is shown as location 422 on the display 426 in FIG. 4). Basedon this information about the gaze 424 of the first user 410 and thelocation information of the representation 430 of the second user 470relative to the display 416, the eye tracking device 422 can determinewhether the gaze direction of the first user 410 is directed to therepresentation 430 of the second user 470 or not (or, similarly, whetherthe gaze location 422 is within or proximate to the representation 430).Alternatively, the communication application can send the locationinformation of the representation 430 to the eye tracking device 422,which can respond with the determination. This determination can be usedto trigger the start and end of the communication as well as alertingthe second user 470. Herein next, the use of the gaze direction isdescribed. Whether the gaze location 422 is within or proximate to therepresentation 430 can be similarly used (e.g., the communication startsif the gaze location 422 is within or proximate to the representation43).

In an example, the communication starts if the gaze direction of thefirst user 410 is directed to the representation 430 of the second user470. Once started, the communication ends if the gaze direction of thefirst user 410 is no longer directed to the representation 430 of thesecond user 470.

In addition, upon determining that the gaze direction of the first user410 is directed to the representation 430, the communication applicationcan send an alert to the second computing device 470 about thisdetermination. In turn, the second computing device 470 may present agraphical and/or audible alert (e.g., via a communication applicationexecuting thereat) about the first user's 410 gaze to the second user460. Similarly, upon determining that the gaze direction of the firstuser 410 is no longer directed to the representation 430, thecommunication application can send an update to the second computingdevice 470 about this determination. In turn, the second computingdevice 470 may present a graphical and/or audible alert (e.g., via acommunication application executing thereat) about the change to thefirst user's 410 gaze to the second user 460.

Furthermore, the above triggers and/or alerts about the communicationmay also be conditioned on the gaze of the second user 470 (e.g., thesecond user's 470 gaze direction or, similarly, gaze location). Inparticular, the second computing device 470 can include similarcomputing components as the ones of the first computing device 420,including an eye tracking device and a communication application.Accordingly, the second computing device 420 can present arepresentation of the first user 410 on the display of the secondcomputing device 420, track the second user's 460 gaze on this display420, and determine whether the second user's 460 gaze direction isdirected to the representation of the first user 410. Information aboutthis determination (e.g., such as a flag set to “1” to indicate that thesecond user's 460 gaze direction is directed to the representation ofthe first user 410 and, otherwise, set to “0”) can be transmitted fromthe second computing device 470 to the first computing device 410. Inturn, the first computing device 420 exchanges the communicationinformation with the second computing device 470 only if both users havegaze directions directed to the respective representations of eachother.

Further, the first computing device 420 includes (e.g., integrates orinterfaces with) an input device 440 such as one or more of amicrophone, a keyboard, a camera, a video camera, or a pointing device.The type of communication with the second computing device 470 dependson the type of the input device. For instance, if the input device 440is a microphone, the communication includes communication data that isvoice data generated by the microphone. If the input device 440 is akeyboard, the communication includes communication data that is textdata generated by the keyboard. If the input device 440 is a camera, thecommunication includes communication data that is an image generated bythe camera. If the input device 440 is a video camera, the communicationincludes communication data that is a video generated by the videocamera.

If the input device 440 is a pointing device, the communication includescommunication data that is a file or document that has been moved on thedisplay 426 to the representation 430 of the second user 470 by means ofthe pointing device. In addition, the communication data comprises theposition of the pointing device. Further, the first computing device canbe configured to transmit the communication data to the second computingdevice 470 only if, additionally, an action is triggered by the pointingdevice during the period when the gaze direction of the first user 410is directed to the representation 430 of the second user 460.

As also illustrated in FIG. 4, the communication server 450 may be usedto manage certain aspects of the communication between the first userdevice 420 and the second user device 470. In particular, thecommunication server 450 can receive management information 452 andcommunication information 454 from the computing devices 420 and 470.Generally, the management information 452 is used to manage the exchangeof the communication information 454 (including any of the digitalcommunication data described herein above). Different types ofmanagement information are possible.

In one example, the management information received from the firstcomputing device 420 includes the determination of whether the firstuser's 410 gaze direction (or, similarly, gaze location) is directed tothe representation 430 of the second user 460. Likewise, the managementinformation received from the second computing device 470 may includethe corresponding determination. Based on either one or bothdeterminations (e.g., that one of the users is gazing towards or on therepresentation of the other user, or both users are gazing on suchrepresentations), the communication server 450 may transfer thecommunication information 454 between the two computing devices 420 and470.

In another, the management information received from the first computingdevice 420 includes the gaze information of the first user 410 and thelocation information of the representation 430 of the second user 460.The communication server 450 may then perform the determination ofwhether the first user's 410 gaze direction (or, similarly, gazelocation) is directed to the representation 430 of the second user 460.Likewise, the management information received from the second computingdevice 470 may include the corresponding gaze and location informationsuch that the communication server 450 performs a similar determinationfor the second user 460. Based on either one or both determinations(e.g., that one of the users is gazing towards or on the representationof the other user, or both users are gazing on such representations),the communication server 450 may transfer the communication information454 between the two computing devices 420 and 470.

In the above embodiments, achieving a directional flow of thecommunication data is also possible. For example, if a determination ismade (e.g., by the first computing device 420 or the communicationserver 450) that the first user's 410 gaze direction is directed to therepresentation 430 of the second user 470 and a determination is made(e.g., by the second computing device 470 or the communication server450) that the second user's 460 gaze is not directed to therepresentation of the first user 410, communication data may flow fromthe first computing device 420 to the second computing device 470, butnot the other way around.

FIG. 5 illustrates a flow implemented by a first computing device tocommunicate with a second computing device based on gaze information,according an embodiment of the present disclosure. A first computingdevice, such as the first computing device 420 of FIG. 4, is describedas performing operations of the flow. Instructions for performing theoperations can be stored as computer-readable instructions on anon-transitory computer-readable medium of this computing device. Asstored, the instructions represent programmable modules that includecode or data executable by a processor(s) of the computing device. Theexecution of such instructions configures the computing device toperform the specific operations shown in FIG. 5 and described herein.Each programmable module in combination with the processor represents ameans for performing a respective operation(s). While the operations areillustrated in a particular order, it should be understood that noparticular order is necessary and that one or more operations may beomitted, skipped, and/or reordered. Further, for the purpose of brevity,the use of gaze directions is described in connection with the exampleflow of FIG. 5. Nonetheless, the example flow can be similarly performedby additionally or alternatively using gaze location.

As illustrated, the example flow of FIG. 5 starts at operation 502,where the first computing device captures a gaze direction of a firstuser of the first computing device. In an example, the gaze direction iscaptured by using an eye tracking device. In particular, the eyetracking device generates gaze information about the gaze of the firstuser on a display of the first computing device. The gaze informationincludes the gaze direction.

At operation 504, the first computing device displays a representationof a second user on the display of the first computing device. In anexample, the representation is presented on a GUI of the display and thefirst computing device tracks a display location of the representation.

At operation 506, the first computing device receives, from the firstuser, communication data generated by an input device. In an example,the first user operates the input device and provides input thereat. Theinput can include text, voice, image, video, file selection,file-related operation, or any other user input depending on the type ofthe input device.

At operation 508, the first computing device determines if the gazedirection of the first user is directed to the representation of thesecond user. In an example, the first computing device compares the gazedirection to the display location of the representation to make thisdetermination.

At operation 510, the first computing device receives, from the secondcomputing device, information regarding if a gaze direction of thesecond user is directed to a representation of the first user on adisplay of the second computing device. In an example, the secondcomputing device may similarly display the representation of the firstuser, track the second user's gaze, and determine whether the gazedirection of the second user is directed to this representation,generate the information accordingly, and transmit this information tothe first computing device over a data network.

At operation 512, the first computing device manages the transmission ofcommunication data to the second computing device based on either one orboth gaze directions (e.g., the first gaze direction of the first useras determined under operation 508 and the second gaze direction of thesecond user as indicated in the information received under operation510). Different types of management are possible. In one example, themanagement depends only on the first gaze direction. In this example,the first computing device transmits the communication data to thesecond computing device, if the gaze direction of the first user isdirected to the representation of the second user. In another example,the management further depends on the second gaze direction. In thisexample, the first computing device transmits the communication data tothe second computing device, only if both the gaze direction of thefirst user is directed to the representation of the second user and thegaze direction of the second user is directed to the representation ofthe first user.

FIG. 6 illustrates a flow for managing communication based on gazeinformation, according an embodiment of the present disclosure. Acommunication server, such as the communication server 450 of FIG. 4, isdescribed as performing operations of the flow. Instructions forperforming the operations can be stored as computer-readableinstructions on a non-transitory computer-readable medium of thiscommunication server. As stored, the instructions represent programmablemodules that include code or data executable by a processor(s) of thecommunication server The execution of such instructions configures thecommunication server to perform the specific operations shown in FIG. 6and described herein. Each programmable module in combination with theprocessor represents a means for performing a respective operation(s).While the operations are illustrated in a particular order, it should beunderstood that no particular order is necessary and that one or moreoperations may be omitted, skipped, and/or reordered. Further, for thepurpose of brevity, the use of gaze directions is described inconnection with the example flow of FIG. 6. Nonetheless, the exampleflow can be similarly performed by additionally or alternatively usinggaze location.

As illustrated, the example flow of FIG. 6 starts at operation 602,where the communication server receives first management informationfrom a first computing device. In a first example, the first managementinformation includes information about a determination performed by thefirst computing device of whether a first gaze direction of a first userof the first computing device is directed to a second representation ofa second user, where the second representation is presented on a firstdisplay of the first computing device. In a second example, the firstmanagement information includes the first gaze direction and displaylocation of the second representation.

At operation 604, the communication server determines if the first gazedirection is directed to the second representation. This determinationmay depend on the type of the received management information. In thefirst example above, the communication server may extract thedetermination of the first computing device from the managementinformation. In the second example above, the communication server mayperform this determination by comparing the first gaze direction to thedisplay location.

At operation 606, the communication server receives second managementinformation from the second computing device. This operation is similarto operation 604, except that this management information is generatedand sent by the second computing device. In particular, the secondmanagement information is generated based on a second gaze direction ofthe second user and a first representation of the first user, where thefirst representation is presented at a display of the second computingdevice.

At operation 608, the communication server determines if the second gazedirection is directed to the first representation. This operation issimilar to operation 606, except that the determination is performedbased on the second management information.

At operation 610, the communication server manages an exchange ofcommunication data between the first computing device and the secondcomputing device based on the first gaze direction and/or second gazedirection. Different types of communication management are possible. Inone example, the management depends only on the first gaze direction. Inthis example, the communication server receives first communication datafrom the first computing device and second communication data from thesecond computing device. The communication server sends the firstcommunication data to the second computing device and the secondcommunication data to the first computing device if the first gazedirection of the first user is directed to the second representation ofthe second user. In another example, the management further depends onthe second gaze direction. In this example, the first computing devicetransmits the first communication data to the second computing deviceand the second communication data to the first computing device, only ifboth the first gaze direction of the first user is directed to thesecond representation of the second user and the second gaze directionof the second user is directed to the first representation of the firstuser.

Watch Together

FIGS. 7-9 illustrate example embodiments for viewing contentsimultaneously on remote devices. In an example, users operatescomputing device to view instances of the same content. Gaze informationof the users can be used to manage the presentation of the contentincluding, for example, the synchronization of the content such that thesame piece of content is presented simultaneously at the displays of thecomputing devices.

FIG. 7 illustrates a computer system for managing the presentation ofcontent based on gaze information, according an embodiment of thepresent disclosure. As illustrated, the computer system includes a firstcomputing device 720 of a first user 710 and a second computing device770 of a second user 760. The first computing device 720 and the secondcomputing device 770 are communicatively coupled over a data network(not shown in FIG. 7), such as the Internet or a local area network, toexchange data usable for managing the presentation of content. Each ofthe computing devices 720 and 770 presents content to the correspondinguser. The content is the same at both computing devices 720 and 770 andis synchronized based on the gaze information such that the same pieceof the content is presented simultaneously by both computing devices 720and 770.

Optionally, the computer system further includes a synchronizationserver 750 that manages certain aspects of the content presentation atthe computing devices 720 and 770. The use of the synchronization server750 is further described herein below and operations thereof are furtherillustrated in FIG. 9.

In an example, the first user 710 operates the first computing device720 to share content 730, such as video, a shared virtual space, audio,video game graphics, text, multimedia pages, a collaborative drawing, orthe like, with the second user 760, where the shared content t is alsopresented on the second computing device 770. The content presented bythe second computing device is shown as content 730′ in FIG. 7 toillustrate that the content 730 presented by the first computing deviceand the content 730′ presented by the second computing device are thesame and are synchronized (e.g., are instances of the same content andthe presentation at each of the computing devices is for the same pieceof the content). Both computing devices 720 and 720 are equipped with aneye tracker, and thus are capable of tracking a user's gaze relative totheir respective displays.

For instance, the first computing device 720 includes a number ofcomputing components, such as a processor, a memory, a display (shown asa display 726 in FIG. 7), input/output (I/O) interfaces, networkinterfaces, and other computing components for receiving input of thefirst user 710 and communicating with the second computing device 770.For instance, the first computing device 720 can be a mobile device,tablet, a laptop, a desktop computer, a wearable device (e.g., a virtualreality device, an augmented reality device; in such a case, the display726 is a head-mounted display) or any other suitable user device. Inaddition, the first computing device 720 includes (e.g., integrates orinterfaces with) an eye tracking device 722 having similar components asthe eye tacking system 100 of FIG. 1. The eye tracking device 722 isused to track a gaze 724 of the first user 710 on the display 726 andaccordingly generate gaze information. The gaze information includes,among other data, a gaze direction of the gaze 724.

In an example, the first computing device 720 executes a contentapplication that facilitates the presentation of the content 730 on thedisplay 726 and the synchronization of the content's 730 presentationwith the presentation of the same content 730′ (e.g., the sharedcontent) by the second computing device 770.

The content application may also interface with the eye tracking device722 (or an eye tracking application executed by the eye tracking device722) based on an application programming interface (API). Accordingly,the content application can receive certain gaze information from theeye tracking device 722 including, for instance, the gaze directionand/or location relative to the display 726 (the gaze location is shownas location 722 on the display 726 in FIG. 7).

Based on the gaze direction and/or the gaze location 722, the contentapplication of the first computing device 720 determines whether thegaze direction of the first user 710 is directed to the display 726and/or falls within the display (or within a GUI that presents thecontent 730).

The second computing device 770 may include similar computing componentssuch as a display, an eye tracking device, and a content application.Accordingly, this content application of the second computing device 770receives gaze information about a gaze 772 of the second user 760,including the second user's 760 gaze direction and/or gaze location 774relative to the display 770. Based on the gaze direction and/or the gazelocation 774 of the second user 760, the content application of thesecond computing device 770 determines whether the gaze direction of thesecond user 760 is directed to the display and/or falls within thedisplay of the second computing device (or within a GUI that presentsthe content 730′). The gaze directions of the users 710 and 720 relativeto the respective displays need not be the same for synchronizing thecontent presentation. Likewise, the gaze locations 722 and 774 of theusers 710 and 720 on the respective displays need not be the same forsynchronizing the content presentation.

In an example, the two computing devices 720 and 770 synchronize theircontent presentations such that both users 710 and 760 are viewing thesame piece of content, based on their gaze activities relative to thedisplays. To do so, upon determining whether the first user's 710 gazedirection is directed to the display 726, the first computing device 720sends an indication of this determination to the second computing device770 over the data network (e.g., such as a flag set to “1” to indicatethat the gaze direction is directed to the display 726 and, otherwise,set to “0”). Similarly, upon determining whether the second user's 720gaze direction is directed to the display of the second computing device770, the second computing device 770 sends an indication of thisdetermination to the first computing device 720 over the data network(e.g., such as a flag set to “1” to indicate that the gaze direction isdirected to the display of the second computing device 770 and,otherwise, set to “0”).

The content presentation is synchronized by using the gaze indications.For example, the first computing device 720 continuously updates thecontent 730 (e.g., keeps playing the video) upon a determination by thefirst computing device 720 that the first user's 710 gaze direction isdirected to the display and the gaze indication received from the secondcomputing device 770 indicates that the second user's 760 gaze directionis directed to the display of the second computing device 770. Likewise,the second computing device 770 continuously updates the content 730′(e.g., keeps playing the video) upon a determination by the secondcomputing device 770 that the second user's 760 gaze direction isdirected to the display of the second computing device 770 and the gazeindication received from the first computing device 720 indicates thatthe first user's 710 gaze direction is directed to the display 726. Ifeither of the conditions are not met (e.g., one of the users 710 and 770is not gazing towards the corresponding display), the contentpresentation is simultaneously paused at both computing devices 720 and770.

Accordingly, if the first user 710 gazes away from the display 726(e.g., their gaze direction is no longer directed to the display 726),the first computing device 720 pauses the presentation of the content730′ and sends an indication of the change to the gaze direction to thesecond computing device 770 over the data network (e.g., updates theflag to “0” and sends the update). Upon receiving this update, thesecond computing device 770 also pauses the presentation of the content730′. Assuming that the gaze direction of the second user 760 has notchanged and is still directed to the display of the second computingdevice 770, upon a return of the first user's 710 gaze towards thedisplay 726, the first computing device 720 un-pauses (e.g., resumes)the presentation of the content 730 and sends another update about thegaze direction (e.g., resets the flag to “1”) to the second computingdevice 770. Accordingly, the second computing device 770 also un-pausesthe presentation of the content 730′. The reverse scenario is likewisepossible, where the second computing device sends updates 770 to thefirst computing device 720 based on changes to the second user's 760gaze relative to the display of the second computing device 770.

As also illustrated in FIG. 7, the synchronization server 750 may beused to manage certain aspects of the content synchronization betweenthe first user device 720 and the second user device 770. In particular,the synchronization server 750 can receive management information 752from the computing devices 720 and 770 and, in response, transmitsynchronization information 754 to the computing devices 720 and 770.Generally, the management information 752 is used to control the pieceof content that should be presented at the computing devices 720 and 770and the timing of the presentation.

In an example, the management information received from the firstcomputing device 720 includes the determination of whether the firstuser's 710 gaze direction (or, similarly, gaze location) is directed tothe display 726. Likewise, the management information received from thesecond computing device 770 may include the corresponding determination.Based on both determinations, the synchronization server 750 may sendthe synchronization information 754 to the two computing devices 720 and770. For example, if both determinations indicate that the two users 710and 760 are gazing towards their corresponding displays, thesynchronization information 754 includes instructions to present thecontent 730 by the first computing device 710 and the content 730′ bythe second computing device 770 at the same time. Otherwise, thesynchronization information 754 includes instructions to pause thepresentation of the content 730 by the first computing device 710 andthe presentation of the content 730′ by the second computing device 770at the same time.

FIG. 8 illustrates a flow implemented by a first computing device tomanage the presentation of content based on gaze information, accordingan embodiment of the present disclosure. A first computing device, suchas the first computing device 720 of FIG. 7, is described as performingoperations of the flow. Instructions for performing the operations canbe stored as computer-readable instructions on a non-transitorycomputer-readable medium of this computing device. As stored, theinstructions represent programmable modules that include code or dataexecutable by a processor(s) of the computing device. The execution ofsuch instructions configures the computing device to perform thespecific operations shown in FIG. 8 and described herein. Eachprogrammable module in combination with the processor represents a meansfor performing a respective operation(s). While the operations areillustrated in a particular order, it should be understood that noparticular order is necessary and that one or more operations may beomitted, skipped, and/or reordered. Further, for the purpose of brevity,the use of gaze directions is described in connection with the exampleflow of FIG. 8. Nonetheless, the example flow can be similarly performedby additionally or alternatively using gaze location on a display.

As illustrated, the example flow of FIG. 8 starts at operation 802,where the first computing device captures a gaze direction of a firstuser of the first computing device. In an example, the gaze direction iscaptured by using an eye tracking device. In particular, the eyetracking device generates gaze information about the gaze of the firstuser on a first display of the first computing device. The gazeinformation includes the gaze direction.

At operation 804, the first computing device determines if the gazedirection of the first user is directed to the first display. In anexample, the first computing device compares the gaze direction (e.g.,in three dimensional space) to a location of the display (e.g., in thethree dimensional space) to perform this determination. Upon a positivedetermination, the first computing device may set a flag to “1,” therebyindicating that the gaze direction of the first user is directed to thefirst display. Otherwise, the flag may be set to “0.”

At operation 806, the first computing device receives informationregarding if the gaze direction of a second user is directed to a seconddisplay. In an example, this information is received over a data networkfrom a second computing device that includes the second display. Thereceived information can include a flag set to “1” to indicate that thegaze direction of the second user is directed to the second display and,otherwise, set to “0.”

At operation 808, the first computing device continuously updatescontent on the first display, if the gaze direction of the first user isdirected to the first display and the gaze direction of the second useris directed to the second display. In an example, the continuous updatesinclude involve presenting pieces of the same content are simultaneouslyon the two displays.

At operation 810, the first computing device receives, from the secondcomputing device, information regarding if the content is paused on thesecond display. In an example, the second computing device sends thisinformation based on a number of factors. One factor may include inputof the second user at an input of the second computing device (e.g., amouse click) pausing the content. Another factor may be a change to thegaze direction of the second user. For instance, if the second usergazes away from the second display, the second computing device maypause the presentation of the content on the second display and may sendan indication of the gaze change (or, similarly, the pausing) in a flagupdate to the first computing device.

At operation 812, the first computing device pauses the content on thefirst display. Various factors can be used for this pausing. In oneexample, the content on the first display is paused if the gazedirection of the second user is not directed to the second display. Inanother example, the content on the first display is paused if thecontent is paused on the second display. In yet another example, thecontent on the first display is paused if the gaze direction of thefirst user is not directed to the first display.

FIG. 9 illustrates a flow for managing to manage the presentation ofcontent based on gaze information, according an embodiment of thepresent disclosure. A synchronization server, such as thesynchronization server 750 of FIG. 7, is described as performingoperations of the flow. Instructions for performing the operations canbe stored as computer-readable instructions on a non-transitorycomputer-readable medium of this synchronization server. As stored, theinstructions represent programmable modules that include code or dataexecutable by a processor(s) of the synchronization server The executionof such instructions configures the synchronization server to performthe specific operations shown in FIG. 9 and described herein. Eachprogrammable module in combination with the processor represents a meansfor performing a respective operation(s). While the operations areillustrated in a particular order, it should be understood that noparticular order is necessary and that one or more operations may beomitted, skipped, and/or reordered. Further, for the purpose of brevity,the use of gaze directions is described in connection with the exampleflow of FIG. 9. Nonetheless, the example flow can be similarly performedby additionally or alternatively using gaze location.

As illustrated, the example flow of FIG. 9 starts at operation 902,where the synchronization server receives first management informationfrom a first computing device. In an example, the first managementinformation includes information about a determination performed by thefirst computing device of whether a first gaze direction of a first userof the first computing device is directed to a first display of thefirst computing device.

At operation 904, the synchronization server receives second managementinformation from a second computing device that is communicativelycoupled with the first computing device. This operation is similar tooperation 902, except that this management information is generated andsent by the second computing device. In particular, the secondmanagement information is generated based on a second gaze direction ofa second user of the second computing device and indicates whether thesecond user's gaze direction is directed to a second display of thesecond computing device.

At operation 906, the synchronization server manages the presentation ofcontent on the first computing device and the second computing devicebased on the first gaze direction and second gaze direction. If bothgaze directions indicate that the two users are gazing towards theircorresponding displays, synchronization server sends synchronizationinformation to the two computing devices, where this informationincludes instructions to present the same pieces of the content by thefirst computing device and the second computing device at the same time.Otherwise, the synchronization information includes instructions topause the presentation of the content by each of the computing devicesat the same time.

What is claimed is:
 1. A method of interaction, performed by a firstcomputing device, comprising: using an eye tracking device, capturing agaze direction of a first user of the first computing device; displayinga representation of a second user on a display of the first computingdevice; receiving from the first user, communication data generated byan input device; determining the gaze direction of the first user isdirected to the representation of the second user; receiving, from thesecond computing device, information regarding a gaze direction of thesecond user is directed to a representation of the first user on adisplay of the second computing device; transmitting the communicationdata to the second computing device, based on determination of both thegaze direction of the first user is directed to the representation ofthe second user on a display of the first computing device, and the gazedirection of the second user is directed to the representation of thefirst user on the display of the second computing device; andtransmitting the communication data to a second computing device of thesecond user, based on determination of the gaze direction of the firstuser is directed to the representation of the second user.
 2. The methodaccording to claim 1, wherein the input device is a microphone and thecommunication data is voice data received from the microphone.
 3. Themethod according to claim 1, wherein the input device is a keyboard andthe communication data is text data received from the keyboard.
 4. Themethod according to claim 1, wherein the input device is a camera andthe communication data is an image received from the camera.
 5. Themethod according to claim 1, wherein the input device is a video cameraand the communication data is a video received from the video camera. 6.The method according to claim 1, wherein the input device is a pointingdevice and the communication data is a file that has been moved on thedisplay to the representation of a second user by means of the pointingdevice.
 7. The method according to claim 1, wherein the input device isa pointing device and the communication data comprises the position ofthe pointing device.
 8. The method according to claim 1, wherein theinput device is a pointing device and wherein the communication dataonly is transmitted to the second computing device if, additionally, anaction is triggered by the pointing device during the period when thegaze direction of the first user is directed to the representation ofthe second user.
 9. A first computing device, comprising: an eyetracking device configured to capture a gaze direction of a first user;and a display configured to display a representation of a second user,wherein the first computing device is configured to: receive from thefirst user, communication data generated by an input device; determinethe gaze direction of the first user is directed to the representationof the second user; receive, from the second computing device,information regarding a gaze direction of the second user is directed toa representation of the first user on a display of the second computingdevice; transmit the communication data to the second computing device,based on determination of both the gaze direction of the first user isdirected to the representation of the second user on a display of thefirst computing device, and the gaze direction of the second user isdirected to the representation of the first user on the display of thesecond computing device; and transmit the communication data to a secondcomputing device of the second user, based on determination of the gazedirection of the first user is directed to the representation of thesecond user.
 10. The first computing device according to claim 9,wherein the input device is a microphone and the communication data isvoice data generated by the microphone.
 11. The first computing deviceaccording to claim 9, wherein the input device is a keyboard and thecommunication data is text data generated by the keyboard.
 12. The firstcomputing device according to claim 9, wherein the input device is acamera and the communication data is an image generated by the camera.13. The first computing device according to claim 9, wherein the inputdevice is a video camera and the communication data is a video generatedby the video camera.
 14. The first computing device according to claim9, wherein the input device is a pointing device and the communicationdata is a file or document that has been moved on the display to therepresentation of a second user by means of the pointing device.
 15. Thefirst computing device according to claim 9, wherein the input device isa pointing device and the communication data comprises the position ofthe pointing device.
 16. The first computing device according to claim9, wherein the input device is a pointing device and wherein the firstcomputing device is configured to transmit the communication data to thesecond computing device only if, additionally, an action is triggered bythe pointing device during the period when the gaze direction of thefirst user is directed to the representation of the second user.
 17. Thefirst computing device according to claim 9, wherein the display is ahead-mounted display.
 18. A non-transitory computer-readable storagemedium storing instructions that, upon execution on a first computingdevice, configure the first computing device to perform operationscomprising: using an eye tracking device, capturing a gaze direction ofa first user of the first computing device; displaying a representationof a second user on a display of the first computing device; receivingfrom the first user, communication data generated by an input device;determining the gaze direction of the first user is directed to therepresentation of the second user; receiving, from the second computingdevice, information regarding a gaze direction of the second user isdirected to a representation of the first user on a display of thesecond computing device; transmitting the communication data to thesecond computing device, based on determination of both the gazedirection of the first user is directed to the representation of thesecond user on a display of the first computing device, and the gazedirection of the second user is directed to the representation of thefirst user on the display of the second computing device; andtransmitting the communication data to a second computing device of thesecond user, based on determination of the gaze direction of the firstuser is directed to the representation of the second user.